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ABSTRACT 

This paper describes selected settings, circumstances, 
problems, and barriers to research in Indian country, and suggests ways to 
mitigate or overcome such problems and barriers. Examples are drawn from four 
research projects. Distinctions are drawn among sponsors of research in 
Indian country (usually federal agencies) ; consumers of such research 
(sponsors, Congress, universities, and other research institutions); and 
stakeholders (sponsors, consumers, tribes, and national and regional Native 
organizations). Federal agencies sponsoring research in Native communities 
and the types of research sponsored are discussed. Sampling issues are 
especially problematic, given the diversity of tribes and their geographic 
dispersion. Problems and approaches to identifying a sample frame and drawing 
a representative sample are examined. Emerging tribal influence on research 
in Indian country is discussed, along with typical tribal requirements of 
researchers. Five such requirements are detailed: hiring tribal members to 
assist in research activities; demonstrating how findings will benefit the 
tribe and tribal members; guaranteeing that the research protocol does no 
harm to the tribe, its members, or the environment; guaranteeing 
confidentiality or anonymity; and publishing results only after tribal review 
and approval. The processes involved in obtaining three levels of approval 
for research are described: approvals by institutional review boards, the 
Office of Management and Budget (OMB), and the tribe. Ethical dilemmas 
confronting researchers in Indian country are discussed. (SV) 
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Abstract 



Selected settings, circumstances, problems, and barriers to research in Indian Country are 
described and ways to mitigate or overcome these problems and barriers are discussed. The 
reemergence of Indian self-determination and self-governance, federal requirements for review of 
research by institutional review boards (IRBs), and the Office of Management and Budget (OMB) 
r; have greatly affected the ways research is conducted in Indian Country. Tribes are gaining 
^ increasing influence over setting research agendas and funding priorities, and have, at a 
r- minimum, the power to reject unwanted research on their lands. Increasingly, researchers are 
P required to identify and justify to concerned tribes the costs, benefits, and risks associated with 
W participation in a study. 



This paper describes selected settings, 
circumstances, problems, and barriers to 
research in Indian Country 1 , and suggests ways 
to mitigate or overcome these problems and 
barriers. Examples of issues, problems, and 
approaches taken from actual research projects 
are presented in four text boxes distributed 
throughout this paper. 

Background 

Stakeholders of Research in Indian Country 

The distinction among sponsors, consumers, 
and stakeholders of research in Indian Country 
is central to the theme of this paper. The 
primary sponsor of research in Indian Country 
is the federal government, with research 
generally conducted for or by federal agencies. 
These agencies may conduct the research 
directly using their own staffs. More often, 
however, agencies fund the research, which is 
conducted, under contract, by other entities 
and organizations. The consumers of research 
conducted in Indian Country include the 
{f) organizations conducting the research, the 
nS sponsoring agencies, Congress, universities, 

cr 



and other research institutions. In recent years, 
consistent with the principles of Indian self- 
determination and self-governance, sponsors 
of research in Indian Country have started to 
relate to Indian tribes and Alaska Native 
villages (and related organizations) as 
consumers of the research findings. 
Stakeholders of research in Indian Country 
include sponsors, consumers, tribes, Alaska 
Native villages, their members, tribal 
consortia, Indian organizations, and Indian 
people who are not members of a tribe. 

Because there are more than 550 Indian tribes 
and Alaska Native villages within the United 
States, national and regional associations, 
organizations, and consortia represent 
important stakeholders of research in Indian 
Country, especially research that includes or 
affects more than one tribe. These Indian 
organizations are often organized around a 
particular area of interest. Examples include 
the National Congress of American Indians 
(NCAI), the National Indian Health Board 
(NIHB), the National Indian Education 
Association (NIEA), and the American Indian 



n 

OS 'Throughout this paper, Indian Country refers to any lands owned by American Indian tribes, Alaska Native villages, and consortia 
_ of tribes or Native villages, and to locations where large numbers of American Indians or Alaska Natives reside. Unless otherwise 

^ indicated, Indian refers to members of American Indian tribes, Alaska Native villages, and their descendants. 
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Higher Education Consortium (AIHEC). In 
addition, most tribes and Alaska Native 
villages are members of consortia or regional 
organizations such as Bristol Bay Native 
Corporation and the Tanana Chiefs Conference 
in Alaska, the United South and Eastern Tribes 
(USET), the All Indian Pueblo Council (AIPC), 
and the Southern California Tribal Chairman's 
Association (SCTCA). 

Federal Agencies Sponsoring Research in 
Indian Country 

Occasionally Congress mandates studies and 
other forms of research in Indian Country; 
however, the bulk of the research is procured by 
or conducted by federal agencies. Most of this 
research is guided by each agency's mission and 
goals; in addition, some research is conducted to 
meet the requirements of the Government 
Performance and Results Act (GPRA) of 1993 
that requires strategic planning and annual 
program performance reporting for every federal 
agency. In passing GPRA, Congress sought 
improved accountability for program 
performance and better planning of federal 
programs. GPRA has resulted in increased 
evaluation and other research to demonstrate 
program results. 

Federal agencies that have conducted or 
sponsored the greatest amount of research in 
Indian Country include the Department of 
Health and Human Services (DHHS), the 
Bureau of Indian Affairs (BIA), the Department 
of Justice (DOJ), and the Department of 
Education (ED). Among these agencies, DHHS 
sponsors the greatest amount of research. 
Within DHHS, research is funded by the Indian 
Health Service (I HS), the National Institutes of 
Health (NIH), the Substance Abuse and Mental 
Health Services Administration (SAMHSA — 
which includes the Centers for Substance 
Abuse Treatment [CSAT], the Center for 
Substance Abuse Prevention [CSAP], and the 
Center for Mental Health Services [CMHS]), 
the Centers for Disease Control (CDC), the 
Administration for Native Americans (ANA), 
and the Administration for Children and 
Families (ACF). 



In the last 10 years, DOJ has become 
increasingly active in promoting research in 
Indian Country. Within DOJ, agencies active 
in conducting or funding research include the 
Office of Justice Programs (OJP), which 
includes the National Institute of Justice (NIJ), 
the Drug Courts Program Office, and the 
Office of Juvenile Justice and Delinquency 
Prevention. Educational research in Indian 
Country is funded by both the BIA and ED. 
Other federal agencies that sponsor research in 
Indian Country include the Departments of 
Labor (DOL), Housing and Urban 

Development (HUD), Commerce (DOC), and 
Transportation (DOT). 

Increasingly, but slowly, federal sponsors of 
research have begun to consult with tribes and 
tribal organizations in setting the agendas and 
guidelines for research to be conducted in 
Indian Country. Congress and federal agencies 
are soliciting input from tribes and tribal 
organizations with respect to research goals 
and research budgets. Some examples of 
agencies would be: 

National Indian Health Board 
http://www.nihb.org/inside_budget.htm 

National Congress of American Indians 
http y/www. nca i .org/ma i n/pages/ 
legislative_updates/index.asp 

United South and Eastern Tribes, Inc. 
http://usetinc.org/resolutions.html 

National Indian Education Association 
http://www.niea.org/2001 niearesos.pdf 

Tribes and tribal organizations track proposed 
legislation and provide information to 
congressional representatives and their staffs 
and to congressional committees. Federal 
agencies often require researchers to establish 
project advisory committees that include tribal 
officials and experts on related research in 
Indian Country. 
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Nature of Research Conducted in Indian 
Country 

The full range of research activities occurs in 
Indian Country, including randomized 
experiments, clinical trials, field experiments, 
quasi-experimental research, retrospective 
and prospective studies, cross-sectional and 
longitudinal studies, survey research, feasi- 
bility studies, evaluation research (including 
outcome, impact, and process evaluations), 
ethnographic research, participant observa- 
tion, and case studies. The types of research 
conducted in Indian Country are not 
distributed evenly across sponsoring agencies. 
Most of the randomized experiments and 
clinical trials are conducted under the auspices 
of the NIH. For methodological (e.g., 
sampling) and logistical reasons, most research 
takes the form of evaluations, particularly 
process evaluations, feasibility studies, and 
case studies. 

Sampling issues are especially problematic for 
research in Indian Country because of an 
implicit political dimension — the heart of 
sampling involves the selection of a subset of a 
defined universe or population. As a rule, it is 
not practical to define the universe population 
of interest as "members of all Indian tribes" or 
"all Indians" because there are over 550 
federally recognized tribes, and there is great 
variation across tribes — variation in language, 
culture, location, economy, and a host of other 
factors. Thus, some researchers have 
discovered something that eludes many non- 
Indian people — American Indians are not a 
homogeneous group; rather, most are members 
of unique tribes. Compounding the sampling 
problem is the large American Indian 
diaspora — more Indians reside in cities and 
other off-reservation locations than on or near 
reservations and other tribal lands (Hillabrant, 
Romano, & Stang, 1992). Because of the great 
diversity across tribes in many domains, 
research findings obtained at one tribe, or on a 
group of tribes, may not be representative of 
other tribes. To plan and draw a representative 



(i.e., random) sample of American Indians 
and/or Alaska Natives residing on or near tribal 
lands would be a daunting task, beyond the 
means and resources of most research projects. 
Identifying a sample frame and drawing a 
representative sample from a single tribe or a 
group of tribes often presents problems that 
challenge the resources available to most 
research projects. Data commonly used to 
identify primary sampling units are often 
incomplete or unavailable. For example, many 
individuals may reside in dwellings without a 
published address or even a named street, lack 
Social Security numbers or telephones, may 
have multiple residences, and may be away 
from their primary residence for long periods of 
time fishing, hunting, trapping, and 
gathering/harvesting. Many of these challenges 
can be attenuated and managed through close 
collaboration with tribal officials and residents 
of the communities participating in the research. 

Of course, it is possible to draw random, 
stratified random, probability proportionate to 
size, and multistage samples of well-defined 
groups in Indian Country. For example, a 
prospective study of substance abuse treatment 
outcomes included a first stage selection of 22 
treatment facilities probability proportionate to 
size (number of clients served the prior year) 
within strata (IHS Areas), and quota samples of 
all clients accepted for treatment at randomly 
determined start dates (Hillabrant, Earp, & 
Rhoades, 2001). In a retrospective study of 
youth residential substance abuse treatment 
facilities, random samples of client records 
were selected for review at each of nine 
facilities (Hillabrant, Earp, & Brutus, 1997). An 
assessment of the expected impact of a change 
in the rule governing eligibility for services from 
the IHS included a random sample of the IHS 
user population — all IHS beneficiaries using an 
IHS or tribal facility in the last 3 years 
(Hillabrant, Earp, & Brutus, 1 996). In this study, 
the sample frame was defined by automated 
patient registration and utilization data. 
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Emerging Tribal Influence on 
Research in Indian Country 

Over the last 25 years, tribes, Native villages, 
and consortia have taken an increasingly 
active role in controlling research conducted 
in Indian Country. This active stance reflects 
the increasing autonomy exercised by tribes in 
taking control over programs such as 
education, the provision of health care 
services, employment, training programs, and 
welfare programs formerly operated by federal 
agencies. With the enactment of the Indian 
Self-Determination and Education Assistance 
Act (ISDEAA) of 1975 (Pub. L. 93-638), tribes 
began to exercise ever-increasing levels of 
control over programs, policies, and research 
affecting their members. 

The increase in tribal control of research has 
been abetted by federal regulations requiring 
protection of people participating in research 
(21 CFR Part 56). These regulations require that 
all research involving human participants be 
reviewed and approved by an institutional 
review board (IRB). Now, some tribes have 
constituted their own IRBs, which review all 
research conducted on or near the reservation 
in which tribal members are participants in the 
research. Regardless of whether or not a tribe 
has instituted an IRB, tribal approval is 
required for any research conducted on tribal 
lands. 

The concern about protection of people 
participating in research, in turn, reflects 
revelation of unethical research in which 
participants were not informed about the risks 
involved or were deprived of treatment and 
care that could have saved their lives or 
ameliorated their conditions. Among the most 
publicized unethical research are 1) a study 
sponsored by the U.S. Public Health Service in 
which 400 African American men in Macon 
County, Georgia, were deprived of medication 
to cure syphilis and not told of their illness, 
putting others at risk of contracting the disease 
(Tuskegee Study); 2) a study conducted by the 
U.S. Army in which soldiers were given the 
drug LSD; and 3) studies where prison inmates 



and terminally ill patients received injections 
of cancer cells to study the response of their 
immune systems (Lemonick & Goldstein, 
2002 ). 

Overcoming History of Exclusion from 
Control of Research 

In the 1 800's, often after successful military 
campaigns and other conflicts with American 
Indians, the United States treated Indian 
people as wards of the federal government 
( Cherokee Nation v. Georgia, 1831 ; American 
Indian Policy Review Commission, 1997). In 
this trustee capacity, federal agencies, often 
acting through the BIA and, later, the IHS (with 
respect to health care), determined what 
research would be conducted, which tribes 
would participate in the research, and which 
tribal members would serve as research 
participants. While the legacy of this trustee 
relationship still reverberates today, tribes have 
increasingly expanded their sovereign control 
over programs affecting their members since 
the passage of ISDEAA in 1975. Because of a 
long history of exploitation and both benign 
and hostile neglect by the United States, many 
tribes and tribal organizations want to be 
involved in all aspects of research in Indian 
Country, including planning, design, data 
collection, reporting of, and dissemination of 
research findings. Tribes and tribal 
organizations are requiring that researchers 1) 
hire tribal members to assist in data collection 
and other research activities; 2) demonstrate 
how the research findings will benefit the tribe 
and tribal members; 3) guarantee that the 
research protocol does no harm to the tribe, 
tribal members, and the environment; 4) 
guarantee confidentiality or anonymity of 
research participants, tribal communities, and 
the tribe; and 5) publish results only after 
review and approval of the manuscript by 
tribal representatives (see e.g., American 
Indian Policy Review Commission, 1997). 
These requirements and limitations increas- 
ingly exercised by tribes can both benefit and 
impede research in Indian Country. 
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1 . Hire tribal members. Requiring researchers 
to hire tribal members to assist in research 
activities can have several benefits. Such 
hiring increases employment and contributes 
to the tribal economy. Tribal members 
assisting with the research acquire skills and 
knowledge. Having tribal members participate 
on the research team is associated with other 
benefits, including: 

a. The research team is less likely to exploit 
research participants, expose them to 
unnecessary risk, or demean them. 

b. Researchers are less likely to employ or 
express invalid stereotypes, or to 
express, explicitly or implicitly, 
prejudicial perceptions, opinions, or 
expectations. 

c. Often, the quality of the data collected is 
improved because of enhanced 
communication and better rapport 
between the researchers and the study 
participants. 

On the other hand, employing tribal members 
on the study team may be associated with a 
variety of costs. Hiring, training, and using 
tribal members on a research team can increase 
both the time required to complete the project 
and project costs. Using tribal members to 
collect data can create special problems for 
study participant confidentiality, and change 
the demand characteristics of the research 
(Orne, 1962). Study participants may conceal 
taboo or socially undesirable expectations, 
beliefs, or behaviors from other tribal members; 
such expectations, beliefs, or behaviors might 
be less concealed from outside investigators 
whom the research participants are unlikely to 
encounter in the future. 

2 . Demonstrate how the research findings 
will benefit the tribe and tribal members. It is 

customary for researchers to describe the 
anticipated benefits (and costs) of the research, 
especially in preparing grant proposals and in 
submissions to IRBs and OMB. Prior to the 
emergence of tribal control of research, 
discussions of the anticipated benefits of 
research in Indian Country tended to be 
abstract, making reference to Indians in 
O 




general rather than to the participating tribe or 
tribes. Requiring researchers to relate costs 
and benefits to the participating tribe(s) tends 
to clarify thinking and make assumptions and 
expectations explicit — results often beneficial 
to the proposed research, the researcher, the 
participating tribe, and other stakeholders. 

On the other hand, research in Indian Country, 
like research everywhere, is conducted in a 
social and political context. The perceived 
benefits and costs associated with a particular 
study may vary across stakeholders. For 
example, managers of tribal programs may 
perceive costs and benefits of a study 
differently from members of the tribal judiciary 
or legislature. Proponents of traditional values 
and approaches may differ from advocates 
of change. While involvement of tribal 
stakeholders with conflicting values, agendas, 
or perspectives can improve the robustness 
and acceptance of a study, such involvement 
can also mire the research in conflict, leading 
to disorganization or even abandonment of the 
research. Depending on their interests, 
stakeholders may embrace or repudiate 
research findings (see Vignette 1). 

3. Guarantee that the research activity does 
no harm to the tribe , tribal members, and the 
environment. The level of risk to study 
participants, the tribe, and the environment 
and the potential to do harm depend on the 
nature of the research, with medical clinical 
research often bearing the greatest risk of harm 
to participants. However, risk is not limited to 
medical research, and all research has costs, 
even if only opportunity costs — time and 
resources spent on the research activity could 
be spent on alternative activities. Opportunity 
costs are especially important in situations 
where there are significant unmet needs, as is 
the case in much of Indian Country. Critics of 
research, including members of tribal IRBs, 
have suggested that most research funds would 
be better spent on service delivery. Of course, 
such judgments should consider the relative 
costs and benefits of a research project. It is 
the responsibility of the research sponsor(s) 
and those conducting the research to present 
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Vignette 1. Repudiation of 
unpopular results: Evaluation of the 
IHS Fiscal Intermediary (FI) 

The IHS commissioned an independent 
evaluation of the performance of the FI in order 
to determine the degree to which the FI was 
achieving the goals and objectives specified in 
its contract. These responsibilities included 1) 
receiving bills and claims submitted by 
hospitals, clinics, and other providers outside 
of the IHS; 2) reviewing the claims to ensure 
conformance with treatment referrals and 
authorizations; 3) paying the valid claims and 
bills; and 4) producing reports of the services 
provided and amounts paid. 

The evaluation was commissioned in the 
context of strong and ongoing criticism of the 
FI by staff in IHS and tribal clinics and by 
tribal officials. The major complaint about 
the FI was that it failed to pay bona fide 
claims within the 45-day time frame 
mandated by its contract with the IHS. Tribal 
officials were especially frustrated because 
providers submitted unpaid bills and claims 
to collection agencies. In turn, collection 
agencies hounded tribal members for 
payment of the claims unpaid by the FI. 

Site visits to the FI and to IHS and tribal clinics 
conducted as part of the evaluation produced 
convincing evidence that the primary causes 
for the delayed payments were incomplete 
referrals and treatment authorizations 
submitted by clinic staffs. As a rule, when the 
FI received complete and valid information 
from clinics, it paid the claims in a timely 
fashion. The evaluation recommended a 
number of ways that the referrals and 
authorizations submitted by the clinics could 
be improved. The evaluation results were 
accepted by many stakeholders; however, 
some, hoping that the study results could be 
used to terminate the FI contract, repudiated 
the findings and ardently tried to suppress them 
(Hillabrant, Brown, Weller, & Earp, 1988). 



the costs and benefits clearly to the tribe, the 
IRBs, OMB, and the study participants. It is the 
responsibility of the tribe, IRBs, and OMB to 
determine if the benefits outweigh the risks, 



that the risks do not exceed acceptable limits, 
and that participants are not exposed to 
unnecessary risks (see Vignette 2). 

There are subtle aspects to the "do no harm" 
dictum for research in Indian Country. Many 
consumers of research in Indian Country may 
be ignorant of 1 ) the history of exploitation and 
discrimination faced by Indian tribes, 
communities, and individuals; 2) the cir- 
cumstances of many tribes (e.g., remote 
location, weather extremes, lack of infra- 
structure); and 3) the legal and constitutional 
status of tribes. Without knowledge of these 
circumstances, such consumers are unlikely to 
appreciate the significance of research 



Vignette 2. Risks involved in a 
screening feasibility study 

Three tribes agreed to participate in a study to 
demonstrate the feasibility of a self- 
administered, multimedia, computer-assisted 
screening system to identify adolescents who 
1) are victims of physical or sex abuse, 2) 
suffer from posttraumatic stress syndrome 
(PTSD), and 3) abuse alcohol and other drugs. 
The system was to be set up at tribal clinics 
providing primary care to tribal members; 
adolescents coming to the clinic for care 
would be recruited to participate in the 
feasibility study. All screening research 
includes the risk of two errors: 1) a false 
positive error occurs when the screening 
system indicates that the adolescent has been 
abused, traumatized, or involved in substance 
abuse when, in fact, no abuse, trauma, or 
substance abuse has occurred, and 2) a false 
negative error occurs when the screening 
system fails to detect abuse, trauma, or 
substance abuse when one or more, in fact, 
occurred. In addition, other risks associated 
with the proposed screening included 1) 
mental health/emotional crisis or problems 
associated with "hits" (correct detection by 
the screening system), and 2) adverse 
consequences should the health care delivery, 
child protection, or other systems fail to 
provide services needed by adolescents 
successfully identified by the screening. 
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findings. While such factors are too numerous 
to list, several favorites of the author include: 

• The unique circumstances of Alaska. Many 
if not most of the people who have never 
visited Alaska do not understand that it is 
farther from Anchorage to half-way through 
the Aleutian Islands than from Washington, 
DC to San Francisco, CA. Most Alaska 
Native villages have a single, unpaved road 
with an airstrip at one end. While there may 
be 20 trucks and cars in the village, they can 
leave the road only in winter when the 
tundra, lakes, and rivers freeze hard. 

• Many residences in Indian Country lack 
electricity, telephones, and running water. 

• For over 100 years, to the present day, the 
United States has been unable to account for 
money and resources held in trust for tribes 
and individual Indians. 

• Tribes are explicitly mentioned in Article I, 
Section 8 of the U.S. Constitution (restricts to 
Congress the power to regulate commerce 
with the tribes), and citizenship was not 
conferred on American Indians until 1924, 
with the passage of the Indian Citizenship 
Act of 1 924. 

•Starting in the late 1800s, some Indian 
children were forced to leave their families 
to attend BIA boarding schools, where non- 
Indians prescribed the language they spoke, 
the religion they practiced, and the material 
they studied. Growing up apart from their 
families, frequently subjected to what is now 
(and often then) seen as physical and 
emotional abuse, had multiple, cross- 
generational negative effects on these 
children, their families, and tribes. While 
some of these boarding schools still exist, 
forced separation of children from their 
families ended, for the most part, by the 
1950s. 

Because many consumers of research in Indian 
Country are ignorant of these and other 
important data, researchers often need to 
provide background information so that the 
research results can be understood in context. 



4. Guarantee confidentiality or anonymity of 
research participants, tribal communities, 
and the tribe. Often, tribes do not object to 
the identification of the tribe or of tribal 
communities in research reports. Other times, 
especially when research is conducted on 
sensitive topics, a tribe may insist that the 
research report not identify the tribe or 
communities participating in the research. 
Protecting the privacy of research participants 
and keeping their identity anonymous can pose 
a special challenge for small tribes and 
communities. If the program being evaluated is 
small, it can be almost impossible to maintain 
the anonymity of key informants who provide 
relevant information (see Vignette 3). Breaches 
of confidentiality and identification of tribal 
communities by researchers, despite their 
agreement not to do so, have been the source of 
frustration and reactions against research by 
several tribes (William Freeman, personal 
communication, June 30, 1999). 

5. Publish results only after review and 
approval of the manuscript by tribal 
representatives. This requirement is often 
imposed by prudent researchers as well as by 
tribes. By obtaining review and feedback from 
the appropriate tribal representatives (generally 
persons who are knowledgeable about the 
research topics), researchers can often identify 
errors or omissions which otherwise might go 
uncorrected. Thus, such review can improve 
the quality of research conducted in Indian 
Country. 

Over the last five years, federal sponsors of 
research have begun to disseminate research 
findings to all federally recognized tribes, tribal 
consortia, and Indian organizations. Both the 
BIA and IHS maintain a "tribal leaders" 
database available to the public and used as a 
mailing list for disseminating research reports 
(see, for example, http://www.ihs.gov/ 
AdminMngrResourcesJndex.asp). Increasingly, 
research reports are being published on the 
World Wide Web, available to tribes and the 
public at large, such as the report from 
Hillabrant, Rhoades, Pindus, and Trutko, 
available at the U.S. Department of Health and 




8 



25 



Vignette 3. Ethical problem: Substance 
abuse treatment center director "using" 

As part of a national evaluation of substance 
abuse treatment outcomes, researchers 
visited 22 treatment programs in Indian 
Country. At one of the programs, a small 
facility with a few staff, several staff members 
told the research team that the program 
director had relapsed and was "drinking 
every day, but without becoming grossly 
inebriated." The staff said that while they did 
not directly observe the program director 
drinking, they noted changes in her behavior 
and claimed to smell alcohol on her breath. 

When asked why they did not report this 
problem to the appropriate tribal authorities, 
the staff said that there was no one at the 
treatment facility who supervised the 
director, and each feared the loss of his or 
her job if they were to report the problem. 
Some staff wanted the researchers to pass on 
the information to tribal officials, but other 
staff, fearing reprisals, asked the researchers 
not to pass on the allegations to others. It 
seemed likely to the researchers that 
reporting the alleged drinking to tribal 
officials could not be done without 
compromising the confidentiality of some of 
the participants (project staff). 

The researchers were unable to independently 
confirm or disconfirm the alleged drinking by 
the program director. Some staff said that the 
program director was able to "remain clean" 
during the two-day site visit. Alternatives 
considered by the researchers included 
ignoring the allegations made by the staff, 
confronting the program director with the 
allegations, or passing on the allegations to 
the office of the tribal chairman. 

The members of the research team decided 
that they would not pass on to tribal officials 
the allegations made by the staff; rather, they 
would contact the agency sponsoring the 
study for guidance. Subsequently, the 
researchers contacted tribal officials and 
discovered that the treatment program 
director had already resigned (Hillabrant, 
Earp, & Rhoades, 2001). 



Human Services website (http://aspe.hhs.gov/ 
hsp/wtw-grants-eval98/tribal02/). 

Participatory, Self-, and Empowerment 

Evaluation 

Program evaluation is supposed to be an 
integral component of program administration 
and improvement. When the program goals 
and objectives are formulated or reformulated, 
the evaluation is to be designed to assess 
progress toward the goals and objectives, and 
the evaluation results are to be used to inform 
and guide reformulation of program goals, 
objectives, policies, and procedures. Too often, 
it seems, the evaluation plans are lost or 
ignored, the evaluation is perceived by 
program managers and staff as a distraction 
from their work, and the evaluation results do 
not have the desired impact on the program. 

In reaction to the underutilization or improper 
utilization of program evaluation, there has 
been a growing interest in conducting 
evaluation research in ways that foster 
improvement and self-determination of 
organizations, communities, and tribes; this 
approach has been termed participatory or 
empowerment evaluation (Fetterman, 2000). 
Empowerment evaluation seeks to help people 
help themselves and improve their programs 
through self-evaluation, reflection, and 
capacity building. Program participants con- 
duct their own evaluations with the aid of an 
outside evaluator who often serves as a coach 
or facilitator. Because of their unique political, 
cultural, and historical circumstances, Indian 
tribes may find the empowerment evaluation 
approach especially useful. 

Approval Processes 

Obtaining the required approvals for research, 
especially in the areas of health and education, 
can be a complex, lengthy, and expensive 
proposition. Often, there are three levels of 
review: IRBs, OMB, and tribal; each is 
discussed below. 



26 



BEST COPY AVAILABLE 



IRBs 

Institutional review boards (IRBs) are 
authorized and required by federal regulations 
(Title 45, Part 46 of the Code of Federal 
Regulations [CFRjas well as 21 CFR Part 50, 
protection of human subjects; 21 CFR Part 56, 
institutional review boards). These regulations 
pertain to research sponsored by the 
Departments of Agriculture, Energy, Com- 
merce, Housing and Urban Development, 
Justice, Defense, Education, Veterans Affairs, 
Health and Human Services, and Trans- 
portation, as well as to research sponsored by 
other federal agencies such as the National 
Aeronautics and Space Administration, 
Consumer Product Safety Commission, Agency 
for International Development, and the 
Environmental Protection Agency. The primary 
objective of IRBs is the protection of human 
research participants. This protection is all- 
encompassing, including physical and mental 
health. Interested readers are referred to the 
Office of Human Subjects Research in the 
DHHS National Institutes of Research 
(http://www.dvincitbt.com/ohsrsite). 

Fundamental to the protection of research 
participants is the concept of "informed 
consent." Informed consent means that the 
study participant understands all the risks and 
rewards associated with participation in the 
research. This understanding is to be 
conveyed to the research participants by the 
research staff in language that the participant 
can fully understand. The research participant 
must indicate his or her understanding of the 
risks and rewards associated with the research 
by signing an informed consent form, and the 
researchers are required to maintain the 
original forms while the research is being 
conducted. 

In Indian Country, there may be multiple levels 
of IRB review, including national (agency- 
wide), area or region, and tribal. While it is 
possible to submit applications simultaneously 
to national and area or regional IRBs, it is not 
uncommon for one group or the other to 
require the prior approval of the other before 
completing their review. Since the IRB may 
O 




meet only once each quarter, obtaining the 
necessary IRB approval may take one year or 
more, especially when one of the IRBs requires 
changes in study protocols, procedures, or 
data collection instruments. Some IRBs 
require the principal investigator to make a 
presentation to the board and to answer 
questions. Prudent researchers include the 
costs of such presentations in their budgets and 
project timelines. 

Obtaining informed consent for research in 
Indian Country involves special challenges. 
The researcher must be sensitive to the ways 
that culture affects the participant's 
understanding and interpretation of the study. 
Some cultural components may seem relatively 
obvious, such as the need for presentation of 
the information in the participant's primary 
language; however, the difficulties in 
translation are often revealed by heated 
discussions among translators as to what 
expression or metaphor should be used to 
communicate key concepts or aspects of the 
research. Other cultural components that affect 
obtaining informed consent and other aspects of 
research in Indian Country might include: 

• Norms about the expression of disagreement 
or conflict; 

• Kinship systems, the perception of how 
individuals are related, and norms governing 
interaction among related individuals; 

• Childrearing practices and associated norms 
and expectations; and 

• Models of health and disease and associated 
norms, expectations, attributions, and 
perceptions. 

Cultural factors that are unknown to the 
researchers can wreak havoc with research — 
"it's what you don't know that can really hurt 
you." One approach to identifying cultural 
factors likely to affect a study is the use of an 
advisory committee that includes key 
informants from the community or 
communities participating in the research, as 
advocated by other authors in this monograph. 
Even with such an advisory committee, 
unexpected cultural factors can damage 
research. In an IHS-sponsored national 
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evaluation of substance abuse treatment 
outcomes for American Indian and Alaska 
Native women, no data were obtained from 
one tribal treatment program because most 
women served by the program refused to 
participate in the study. Many of these women 
were angry because 1 ) they had been forced to 
enter into treatment by a judicial proceeding, 
and 2) men were not required (or allowed) to 
participate in the study. While these 
circumstances were not unique to the tribe in 
question, they interacted with the unique 
aspects of the tribe , s culture pertaining to 
gender roles; the other 21 tribal programs in 
the study successfully recruited into the study 
women in similar circumstances (Hillabrant, 
Earp, & Rhoades, 2001). 

Another aspect of research participant 
protection is the maintenance of con- 
fidentiality of participant data. Researchers 
must demonstrate to the IRB that no one, 
except the researchers, will be able to obtain 
or infer information about individual 
participants collected during the research. 
Researchers working in the area of substance 
abuse can face special challenges to 
maintaining participant confidentiality. 
Because substance abuse can include the use 
of illegal drugs (e.g., marijuana, cocaine, 
heroin, or the use of prescription drugs without 
a legitimate prescription), law enforcement 
officials may request study data and, in 
particular, the names of research participants 
who use illegal drugs. Such requests can come 
in the form of a subpoena issued by a court. 

A Certificate of Confidentiality , issued by the 
National Institute on Alcohol Abuse and 
Alcoholism (NIAAA), authorizes researchers to 
withhold the names and other identifying 
characteristics of people who participate as 
subjects in the research project specified in the 
certificate. This authorization applies to all 
persons who, in the performance of their duties 
in connection with the research project, have 
access to information that would identify the 
subjects of that research. Persons so 
authorized may not, at any time, be compelled 



to provide the names or other identifying 
characteristics of the research participants 
encompassed by the certificate in any federal, 
state, or local civil, criminal, administrative, 
legislative, or other proceedings. The statutory 
authority for the certificate of confidentiality is 
section 301 (d) of the Public Health Service Act 
(part USC 241 [d]). The applicable regulations 
are set forth at 42 CFR Part 2a. 

OMB Review 

The Paperwork Reduction Act of 1995 requires 
that the Office of Management and Budget 
(OMB) review and approve all federally 
sponsored research, other than grants, that 
includes more than nine individuals. The 
materials are submitted to the OMB by the 
funding federal agency; however, contractors 
and researchers often draft the "OMB 
submission package" for the client agency. 
The OMB package includes: 

• Justification of the research, including 
proposed use of the study findings; 

• Explanation of information technology used 
to promote efficiency and reduce the burden 
of the research participants; 

• Explanation of how participant con- 
fidentiality will be assured; 

• Estimates of burden on research participants 
and cost of the research; 

• Project schedule and publication plans; 

• Statistical methods to be used (generally 
includes sample design, research design, and 
planned analyses); 

• Notices describing the study to be published 
in the Federal Register. 

The process of preparing the OMB submission 
package, responding to questions raised by the 
OMB, and obtaining the required approval or 
clearance generally requires 6-12 months. In 
the past, some agencies have taken the stance 
that research conducted in Indian Country is 
exempt from OMB review because of tribal 
sovereignty. The position taken is that the 
research is being conducted on behalf of the 
tribes, and the tribes are not subject to the 
Paperwork Reduction Act. Such research has 



proceeded without obtaining an OMB 
clearance number and, presumably, without 
OMB's knowledge. 

Tribal Approval 

Whether or not a tribe operates an IRB, 
research on tribal lands requires tribal 
approval. If the tribe has delegated 
responsibility for review and approval of 
research proposals to an agency or office, the 
review process is greatly simplified. If no such 
delegation exists, the tribal council, business 
committee, or other group reviews the research 
proposal; such review can be a lengthy 
process. The approval process can be 
expedited, and the quality of the research 
improved, if the researchers identify and 
persuade stakeholders at the tribe of the merits 
of the study. 

An advocate for the research who works for or 
with each tribe can make the difference 
between a long review process ending in 
disapproval, and an expedited, successful 
review. For example, one tribe approved its 
participation in the national welfare-to-work 
(WtW) evaluation within four weeks and 
approved participation in the women's 
substance abuse treatment outcome study only 
after two years of deliberations (Hillabrant, 
Rhoades, Pindus, & Trutko, 2001; Hillabrant, 
Earp, & Rhoades, 2001 ). While the two studies 
varied across many dimensions, two critical 
factors seemed to account for the difference in 
speed of approvals: 1) approval of the tribal 
IRB was required in the treatment outcome but 
not in the WtW evaluation, and 2) the WtW 
evaluation had the strong support of key 
stakeholders (heads of tribal departments) 
whereas the treatment outcome study lacked a 
strong advocate at the tribe. Interestingly, the 
treatment outcome study was supported by 
some stakeholders; the substance abuse 
treatment programs supported the study, but 
corresponding support was lacking among 
stakeholders with the needed "influence." 

Sometimes stakeholders at tribes are 
enthusiastic about participating in a research 



project because they want to share successful 
approaches with other tribes. For example, in 
a study of family violence in Indian Country, 
one tribe had developed a model approach 
that included legislation identifying family 
violence as an offense and implementation of 
effective policies and procedures for tribal 
police and courts — mandatory arrest of 
perpetrators, use of male-female police teams 
responding to family violence complaints 
(Hillabrant, Brown, Colhoff, & Earp, 1994). In 
the WtW evaluation, a tribe had developed a 
model approach to child support enforcement 
(CSE) which involved close collaboration with 
states to identify noncustodial parents who 
were employed but were not providing support 
for their children and to garnishee the wages of 
persons who refused to comply with tribal 
court orders to provide child support 
(Hillabrant, Rhoades, Pindus, & Trutko, 2001). 

Longitudinal research and studies that involve 
data collection for six months or more can face 
additional problems including changes in 
tribal administration and changes in program 
staffing. Changes in tribal administration (e.g., 
new chairman, chief, governor) or newly 
elected tribal council members may view with 
suspicion research approved by a prior 
administration. Soliciting and obtaining 
approval of proposed research from a wide 
range of stakeholders can circumvent or 
minimize the effects of changes in 
administration. Effects of changes in program 
staffing on a study can range from minimal to 
catastrophic, depending on the nature of the 
research. Frequently, new staff must be trained 
to use or interface with the study protocol. 

Ethical Problems and Issues 

In general, research in Indian Country may 
have neither more nor fewer ethical problems 
and dilemmas than research conducted 
elsewhere. Still, issues such as cultural 
competence, relatively high rates of poverty 
and illness, and associated deficits in 
infrastructure (e.g., public transportation, 
plants and facilities, telecommunications) can 
exacerbate ethical problems and dilemmas. 



University graduate training programs teaching 
research ethics may cover federal regulations 
and procedures associated with IRBs and OMB 
as well as regulations governing the care of 
animals in research (Health Research 
Extension Act of 1985); however, it is difficult 
for such training to focus on ethical issues and 
problems associated with persons who might 
come into contact with the research but who 
are not participants. Also, it is difficult for 
ethics training to address problems where the 
researcher's actions must balance immediate 
costs against potential greater costs. This 
paper concludes with discussion of such a 
problem. 

Already discussed in Vignette 3 is the 
allegation of on-the-job substance abuse by a 
treatment program director. Another example 
involves the evaluation of a Head Start 
program producing exceptional benefits that 
was housed in a building that represented 
unacceptable risks to the children it served (see 
Vignette 4). The evaluators were confronted 
with the choice of reporting the violations, 
likely to result in the suspension of the 
program, or passively acquiescing in the 
continued exposure of the children to the risks 
associated with an unsafe building. 

Making judgments about ethics and values can 
be challenging to researchers working in 
Indian Country because of potentially 
conflicting roles and circumstances, such as: 

• If the researcher is not a member of the tribe 
participating in the study, he or she may be 
unfamiliar with critical values and 
expectations pertaining to ethical issues. 

• The sponsor of the research may have 
agendas, rules, and expectations that are 
different from or in conflict with those of the 
tribe(s) participating in the study. 

• The protection of study participant 
confidentiality may conflict with protection 
of study participants from abuse, family 
violence, or other threats. 

In such circumstances, it is prudent for the 
researcher to seek guidance from the project 
advisory committee, the research sponsor, 



and/or legal authorities without disclosing 
information that would violate the identity of 
the research participant(s) or violate the 
confidentiality of participant data. 



Vignette 4. Ethical problem: Closing an 
excellent Head Start facility. 

The national Head Start program conducts 
rolling compliance reviews of local Head Start 
facilities to ensure that the programs protect 
the health and welfare of the children served. 
One of the program requirements is that no 
Head Start facility be located in a building 
with major building code violations. 
Conducting site visits at several facilities on a 
reservation, researchers found, with one 
exception, that each facility met or exceeded 
all Head Start program requirements. The 
children attending these programs were 
clearly flourishing, and entire families were 
benefiting from the program. One facility, 
however, was located on the second story of a 
wooden building that had multiple building 
code violations, including insufficient 
numbers of fire extinguishers and smoke 
detectors. Program staff were aware of the 
code violations; however, there were few 
alternate facilities available on the reservation 
that could be used by the Head Start program, 
and tribal officials had not responded to 
requests for better facilities. 

The researchers did not want to jeopardize 
the operation of the clearly successful 
program. Options discussed included ignor- 
ing the code violations, urging program staff 
to continue their efforts to secure an alternate 
facility, and reporting the infractions to Head 
Start headquarters in Washington. 

After a brief discussion, the study team 
concluded that failure to act, in this case, 
would subject the children participating in the 
program to unacceptable risks. Consultation 
with Head Start headquarters resulted in an 
order that the tribe immediately suspend 
operation of the program at the dangerous 
facility. This suspension stimulated tribal offi- 
cials to action. Within a week, the tribe was 
able to make available a building that 
complied with Head Start regulations and the 
program resumed operations. 
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